1. Field of the Invention
The present invention generally relates to the field of image processing, and specifically, a method and an apparatus for separating a foreground image from an image.
2. Description of the Related Art
The technology of foreground image separation, i.e., the technology of image cut-out means extracting or separating an object from an image and estimating the transparency, i.e., Alpha-value (α masking) of each pixel in a region covered by the object. An extracted object region is referred to as a “foreground region” or a “foreground image” of an image, and other portions are referred to as a “background region” or a “background image” of the image.
Such technology is widely applied in image or video editing applications. A user can separate foreground by this technology and combine the foreground with another background. As an example, in virtual photographing, the foreground such as a person can be combined with the virtual background to obtain an immersive experience, and for example, a conference participant can be separated from the original background in a video conference and be combined with another background.
From the viewpoint of image composition (image combination), an image I(x,y) may be modeled as a linear combination of a foreground image F(x,y), a background image B(x,y) and a α (x,y) value, as shown in the following formula (1).Ip=αpFp+(1−αp)Bp  (1)
In which Ip is a pixel value of a pixel p of an image, Fp is a foreground component of the pixel p, Bp is a background component of the pixel p, αp is an Alpha-value of the transparency and the range of αp is [0,1]. When αp=1, the pixel p belongs to a foreground region, and when αp=0, the pixel p belongs to a background region.
That is to say, an image cut-out problem becomes a problem to solve the above equation. Therefore, a three-color image is usually used as a limitation of an additive condition to solve the equation. The three-color image includes marked images of three colors, i.e., black, white and gray images, where each color represents a region type. The black represents a known background region, the white represents a known foreground region, and the gray presents an unknown region, in which regions belonging to the background and regions belonging to the foreground cannot be determined.
Because of the introduction of a three-color image, solving of an image cut-out becomes calculating an Alpha-value, a foreground component Fp and a background component Bp of unknown pixels p (rather than all pixels of a whole image). Fp and Bp can be obtained by performing estimation based on adjacent points of the pixel p. After obtaining Fp and Bp, Alpha-value αp of the pixel p can be obtained according to the following formula (2).
                              α          p                =                                            (                                                I                  p                                -                                  B                  p                                            )                        *                          (                                                F                  p                                -                                  B                  p                                            )                                                                                                          F                  p                                -                                  B                  p                                                                    2                                              (        2        )            
From the above, the forming of a three-color image is the basis of image cut-out processing, and a more accurate three-color image helps to separate a foreground object from an image or a video more accurately.
In the current methods for forming a three-color image, a three-color image in an image may be specified by a user in a user-interaction way, and for example, to draw some line segments in an image to specify which part is the foreground and which part is the background in the image. In this method, it is necessary for the user to generate the three-color image manually, thus it takes user's labor and the generated three-color image is not accurate enough. Depth information may also be introduced to help to automatically generate a three-color image. In this method, user interaction is not required, however an accurate three-color image still cannot be generated in some cases. For example, in a case where a foreground region includes only one color and the color does not appear in other foreground regions, the region may be marked as an unknown region, and as a result, the region may be classified as a background region incorrectly.
Furthermore, as described above, calculation of an Alpha-value of a unknown pixel p relies on sampling of known foreground points and background points in an adjacent region of the pixel p, therefore obtaining more accurate foreground sample points and background sample points is very important for calculation of Alpha-values, e.g., the quality of cut-out.